Glossary from papers

15 Words and expressions from articles. terms

configuration
noun
An arrangement of parts or elements in a particular form, figure, or combination.
Example: A “configuration” of a machine learning model is a unique set of values for different hyperparameters of the model.
ru: конфигурация
discarding
noun
getting rid of (someone or something) as no longer useful or desirable
Example: These algorithms rely on probability theory to determine how many times each arm must be pulled in order to decide if the arm is suboptimal and should no longer be pulled, either by explicitly discarding it, e.g., Successive Elimination (Even-Dar et al., 2006) and Exponential Gap Elimination (Karnin et al., 2013), or implicitly by other methods, e.g., LUCB (Kalyanakrishnan et al., 2012) and Lil’UCB (Jamieson et al., 2014).
ru: отбрасывание
evaluation
noun
the making of a judgement about the amount, number, or value of something; assessment
Example: Bayesian optimization techniques model the conditional probability p(y| lambda) of a con guration's performance on an evaluation metric y (i.e., test accuracy), given a set of hyperparameters lambda.
ru: оценка
exhibit
verb
manifest clearly (a quality or a type of behaviour)
Example: These algorithms also exhibit some undesirable behavior with respect to how many losses they observe, which we explore next. While these algorithms were developed for and analyzed in the stochastic setting, they exhibit attributes that are very amenable to the non-stochastic setting.
ru: (наглядно) показывать
gradient
noun
In vector calculus, the gradient is the vector field whose value at a point is the "direction and rate of the fastest increase".
Example: Under a rich dataset model, we show that gradient descent is provably robust to noise/corruption on a constant fraction of the labels.
ru: градиент
halving
noun
Reduction by half.
Example: In another line of work, Sparks et al. (2015) proposed a halving style bandit algorithm that did not require explicit convergence behavior. The idea behind the original Successive Halving algorithm follows directly from its name: uniformly allocate a budget to a set of hyperparameter configurations, evaluate the performance of all configurations, throw out the worst half, and repeat until one configuration remains.
ru: деление пополам
hyperparameter
noun
In machine learning, a hyperparameter is a parameter whose value is used to control the learning process (by contrast, the values of other parameters are derived via training).
Example: Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. In hyperparameter optimization for supervised learning, we are given labeled training data, a set of hyperparameters associated with our supervised learning methods of interest, and a search space over these hyperparameters.
ru: гиперпараметр
manifold
noun
a collection of points forming a certain kind of set, such as those of a topologically closed surface or an analogue of this in three or more dimensions
Example: Our starting point is the geometric perspective on representation learning through the lens of the manifold hypothesis (Goodfellow et al., 2016), according to which real-world data presented in a high-dimensional space are expected to concentrate in the vicinity of a manifold of much lower dimension.
ru: многообразие
optimization
noun
An act, process, or methodology of making something (such as a design, system, or decision) as fully perfect, functional, or effective as possible. Specifically : the mathematical procedures (such as finding the maximum of a function) involved in this.
Example: As supervised learning methods are becoming more widely adopted, hyperparameter optimization has become increasingly important to simplify and speed up the development of data processing pipelines while simultaneously yielding more accurate models. We formulate hyperparameter optimization as a pure-exploration nonstochastic infinite-armed bandit problem where a predefined resource like iterations, data samples, or features is allocated to randomly sampled configurations.
ru: оптимизация
overfitting
noun
The production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably (In mathematical modeling).
Example: There are basically two ways to fight overfitting: reducing the number of dimensions of the parameter space or reducing the effective size of each dimension.
ru: переобучение
phenomena
noun
A fact or situation that is observed to exist or happen, especially one whose cause or explanation is in question.
Example: This paper focuses on an intriguing phenomena: overparameterized neural networks are surprisingly robust to label noise when first order methods with early stopping is used to train them.
ru: феномен
posterior
noun
posterior probability distribution
Example: Additionally, without structural assumptions on the covariance matrix of the GP, tting the posterior is O(n3) (Wilson et al., 2015).
ru: апостериорное распределение
representations
noun
In mathematics, a representation is a very general relationship that expresses similarities (or equivalences) between mathematical objects or structures.
Example: We propose a method for comparing two data representations.
ru: представление
similarity
noun
The state or fact of being similar, a similar feature or aspect,.
Example: This ensures that our score has scaling invariance, the reasonable property of a good representation similarity measure.
ru: сходство, подобие
supervised
adjective
under observation or under the direction of a superintendent or overseer in machine learning: supervised learning - learning with given answers
Example: Validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ("early stopping").
ru: контролируемое